Notional  Joint  Strike  Fighter: 
Visualization  and  Mapping  Data  within  the 
Virtual  Strike  Warfare  Environments 


Floyd  Adagio 

Pacific  Sierra  Research  Corp.,  Arlington,  VA 

CDR  Jon  Lawler,  RN 

Joint  Strike  Fighter  Program  Office,  Arlington,  VA 


ABSTRACT 

Various  levels  of  the  military  command  infrastructure  require  visualization  of  data  and  fusion  of 
tactical  and  strategic  information.  Specifically,  visually  oriented  displays  can  provide  intuitive, 
readily  understandable  information  that  is  easily  interpreted  and  acted  upon.  Visual  systems 
provide  a  variety  of  data  and  information,  including  key  elements  such  as  maps,  terrain 
elevation,  imagery,  iconic,  symbology,  and  text.  Use  of  these  information  forms  must  be 
specific  to  individual  mission,  yet  consistent  to  all.  Consistency  among  and  between  command 
levels  is  critical  to  support  coordinated  planning,  execution,  and  after-action  activities. 
Information  forms  and  presentation  to  a  user  are  constrained  by  system  and  cost  considerations. 
This  manuscript  discusses  the  application  of  virtual,  visualization  environments  within  a 
hierarchical  requirements  structure  based  on  warfighting  functional  requirements. 


1.0  INTRODUCTION 

The  ability  to  convey  information  and  to  exploit  it  is  an  age-old  issue.  Plato  referred  to  the 
purest  of  information  as  a  form  (Grube,  1974).  He  also  discussed  mental  processes  that  are 
appropriate  to  military  visualization  within  a  continuum  of  mental  processes,  including 
understanding,  reasoning,  opinion,  and  imagination.  The  philosopher  discussed  his  notion  of 
forms  as  the  highest  level  of  knowledge.  In  the  present  as  well  as  the  past,  the  ability  to  impart 
the  highest  level  of  knowledge  is  critical  to  the  human  endeavor,  whether  in  war  or  peace.  Thus, 
the  objective  of  providing  the  perfect  information  should  strive  to  provide  understanding  by 
intuitive  means  at  the  form  level  of  the  knowledge  continuum. 

Today’s  visualization  of  military  forms  presents  unique  challenges  and  perplexing  technological 
trade-offs.  Achieving  the  right  "look  and  feel"  for  combat  data  is  difficult.  A  gap  in 
communication  and  interpretation  exists  between  users  and  developer.  Decision  makers  of  the 
slide  rule  generation  are  often  determining  system  needs  for  operators  who  grew  up  with  video 
games  and  personal  computers. 


2.0  DISCUSSION 

Visualization  requirements  span  a  continuum  from  simple  line  depiction  to  fusion  of  multiple 
sources  of  data  and  rendering  of  high  fidelity,  geo-specific  perspective  scenes  in  real-time.  The 
Joint  Strike  Fighter  (JSF)  Program  has  employed  a  Virtual  Strike  Warfare  Environment  (VSWE) 
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to  develop  and  refine  requirements  within  an  affordable  and  realistic  modeling  and  simulation 
environment.  This  range  of  capabilities  has  been  employed  to  support  mission  planning, 
rehearsal,  execution,  debrief,  and  intelligence  efforts.  Future  VSWE  efforts  will  support  Test 
and  Evaluation  as  well  as  Engineering  and  Manufacturing  Demonstrations. 

Several  areas  are  impacted  by  visualization  enhancements,  as  provided  within  US  Defense 
Science  and  Technology  Guidance  for  Joint  DOD  applications: 

♦♦♦  Command,  Control,  Communications,  and  Computers  (C'4) 

■  Provide  common,  accurate,  mission-tailored  picture  to  all  warfighters 

■  Develop  ability  to  “learn  ’’from  users 

■  Improve  visualization 

♦>  Intelligence,  Surveillance,  Reconnaissance  (ISR) 

■  Improve  situational  awareness 

■  Manage  information  and  decision  aids 

❖  Manpower  /  Personnel 

■  Determine  optimum  human/computer  tasking 

■  Improve  decision-making 

■  Reduce  user  workload 

■  Reduce  crew  /  manning  requirements 

These  capabilities  will  foster  warfighter  performance  improvements  and  manpower  reductions 
by  tailoring  displays  to  user  requirements. 

Consistent  and  repeatable  visualization  capabilities  are  critical  throughout  and  among  command 
levels.  Examples  of  advanced  applications  for  concurrent  visualization  at  multiple  levels  include 
recent  activities  for  Yugoslavian  Operations:  planning  and  rehearsal,  exploitation  within  the  JSF 
development  program,  investigation  of  US  Secretary  of  Commerce  Brown’s  crash,  and 
conveying  data  to  US  National  Command  Authority.  Simulated  environments  offer  the 
opportunity  to  “see  into  the  future”  and  allow  operators  to  expand  their  understanding  of  future 
events  while  expanding  situational  awareness.  These  virtual  environments  also  provide  user- 
augmented  reality  with  actual  situations,  which  may  have  limited  information  resources.  An 
example  is  evident  in  airborne  applications  to  enable  flexible  targeting  and  re-tasking,  as  well  as 
a  context  for  off-board  information  fusion. 

The  Tactical  Aircraft  Moving  Map  Capability  (TAMMAC)  system  is  planned  to  support  two- 
dimensional  (2D)  map  requirements  for  naval  aviation  (table  1).  Figure  1  provides  an  example 
of  desired  2D  rendering  capabilities  for  the  TAMMAC.  Planned  TAMMAC  improvements 
include  implementation  of  3D  functions.  3D  perspective  scene  generation  needs  were  assessed 
in  terms  of  user  suggested  enhancements  to  F-18  and  desired  JSF  capabilities. 


Table  1.  2-D  Requirements  Based  on  the  US  Navy  TAMMAC  (Lohrenz,  et  a!.,  1997). 


Change  map  scales  and  features  in  real-time 


Magnify  /  Zoom  from  1  to  2x  in  steps 
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Coverage  area  in  standard  scales  (e.g.,  1:500,000) 

Programmable  color  palettes 

Variable  map  scales  between  2.5  -  250  nm 

Standard  symbol  generation 

Map  orientation  north-up  or  heading-up 

Dynamic  overlays 

Center  /  de-center 

Data  frame  rates 

Sun  /  moon  shading 

Slewing 

Figure  1.  Example  of  2D  map  display  similar  to  2D  TAMMAC  map. 


Table  2  provides  a  list  of  3D,  scene  perspective  needs  related  to  the  JSF  STT  framework.  Users 
have  asked  that  future  visualization  also  perform  the  following  functions: 

■  Incorporate  other  sensor  imagery. 

■  Simulate  sensor  and  performance  predictions. 

■  Extrude  objects  and  buildings. 

■  Provide  aircraft  flight  dynamics,  weapon  simulations,  dynamic  and  accurate  threats. 

■  Provide  an  interface  to  mission  planning. 

■  Permit  users  to  retrieve  coordinates. 

Figure  2  provides  a  notional  view  of  a  perspective  scene  using  photo-based  imagery,  elevation 
data,  and  appropriate  strike  symbology  (3D  threats  and  target  locations,  approach  restrictions, 
sensor  updates  and  weapons  effects).  The  2D  information  portrayed  in  Figure  1  illustrates  a 
display  that  requires  interpretation  by  the  user.  The  information  in  Figure  2  provides  intuitive 
graphics  in  mission  context  for  a  strike  platform.  Additionally,  the  growth  of  communications 
and  display  techniques  provides  an  opportunity  to  update  this  information  within  the  mission 
timeline,  providing  expanded  sensor  data  to  the  warrior.  The  intent  is  to  provide  Common 
Battlefield  Awareness  that  provides  command,  control,  and  operational  data  to  the  user  within 
the  mission  context. 
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Table  2.  3-D  Requirements  Based  on  the  JSF  Virtual  Ground  Map  Functionality. 


Multi-source  /  spectral  data 

Threat  overlays 

Multi-source  /  resolution  elevation 

Updated  and  real-time  threat  overlays 

Geo-located/specific  imagery  and  elevation 

Multiple  Fields  of  view 

Display  of  vector  products  /  data 

Degrees  of  Freedom  view  angle  control 

Annotations  and  pointers 

Remote  and  slewable  eye  point. 

Figure  2.  Notional  view  of  perspective  scene  using  photo-based 
imagery,  elevation  data,  and  appropriate  strike  symbology. 


The  utility  of  this  data  can  be  seen  at  various  levels,  as  depicted  in  figure  3.  During  the  JSF 
VS  WE  Event  #4  held  in  Mesa,  AZ  (spring,  1999),  two  levels  of  visualization  included  2D  and 
3D  scene  generation.  Figure  4  provides  an  overview  of  the  capabilities  provided  within  the 
VSWE  #4  cockpit.  A  video  of  this  effort’s  goals  and  objectives  was  developed  and  the  event’s 
findings  were  included. 

Preview,  rehearsal,  and  data  fusion  (of  intelligence  and  threat  data)  were  performed  at  the 
mission  level  by  providing  enhanced  command  and  control  using  an  intuitive  display. 
Additionally,  "what  if'  scenarios  can  be  played  using  advanced  modeling,  simulation,  and 
analysis  tools,  which  can  be  linked  to  higher  level  analysis  using  the  JSF  STT  framework. 
Tactical  information  can  be  fused  into  scenes  appropriate  to  a  design  of  experiments  and  for  real¬ 
time  simulation  use  of  data  from  multi-spectral  databases,  including  elevation  data,  imagery,  and 
advanced  data-links  provided  in  on-board  databases  fused  with  off-board  resources.  Data  was 
also  overlaid  and  updated  via  Intel  sources. 
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INTERACTIVE  SIMULATIONS  CON  OP  S/TACTICS  DEVELOPMENT 


GENERIC  A/C  PERFORMANCE  SYNTHETIC  BATTLESPACE  COCKPIT 


Figure  3.  Virtual  Strike  Warfare  Environment  (VSWE)  functions. 


Operational  level  data  was  passed  further  to  individual  mission  areas  and  exploited  at  lower 
levels  by  individual  operators  within  a  specific  mission  context.  Integrated  tactical  displays 
linked  to  information  resources  at  unit  and  command  levels  could  provide  current  operational 
information  beyond  that  available  today.  Such  capabilities  could  enable  flexible  re-targeting  and 
re-planning,  in-flight  updates  with  data  and  imagery,  remote,  future  views  within  virtual  reality 
concepts.  Operational  concepts  might  include  video  teleconferencing.  At  the  mission  level, 
virtual  walk-through  and  fly-through  capabilities  would  allow  users  to  see  through  the  next  hill 
or  discuss  safe  transit  routes.  Interfaces  to  mission  planning  resources  could  provide  the  view 
ahead  and  support  position  and  maneuver  control,  visually  augmented  identification  of  friend  or 
foe,  as  well  as  obstacle  identification  within  tactical  situation  displays.  Opening  the  operator's 
line  of  sight  by  projecting  an  outside  view  through  the  fuselage  of  an  aircraft  could  provide  a 
better  understanding  of  the  object  in  a  larger  context. 

Two  important  principles  that  should  guide  future  visualization  developments  are  (1)  an 
emphasis  on  using  real-world  photo-imagery,  and  (2)  the  use  of  portable,  standardized  software 
tools  that  do  not  require  non-standard  hardware.  Additionally,  military  and  commercial  systems 
that  support  visualization  should: 

■  Use  and  display  real  world  data,  from  any  perspective,  and  in  any  desired 
level  of  detail. 

■  Exploit  imagery  of  varying  scales,  breadths,  or  resolutions  from  satellites, 
aerial  photographs,  and  other  sources. 
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Combine  the  multi-source  imagery  with  other  data  (such  as  maps  and  cultural 
features)  into  a  seamless  visual  scene. 

Correlate  the  scene  to  real-world  coordinates. 


ATCS  Pilot  Workload  Assessment 


Main  instrument  Panel  Configuration 


HUD /HMD  Display  on  Dome 
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SAR  Map 
Targeting 

Perspective 
Target  Imagery 
(Off board  Data 
interface) 

Fused  Tactical  Situation  Display 


-  Twin  Large  Color  Flat  Panels 

-  Display  Windowing 

-  Color  Upfront  Control 

-  Digital  Map 


Figure  4.  VS  WE  #4  cockpit  capabilities. 


3.0  SUMMARY 

Future  visualization  techniques  for  military  users  include  requirements  to  provide  intuitive  2D 
and  3D  data  within  a  mission  context.  Present  graphics  techniques  are  expanding  to  provide 
consistency  within  military  levels  and  individual  operator  needs  and  to  provide  greater 
opportunity  within  the  C4I  context.  The  users  of  these  integrated  techniques  will  foster  the 
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development  of  enhanced  operational  concepts  and  future  operations  that  will  improve  the 
military's  abilities  to  reach  strategic  and  operational  goals. 
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